Bilingual Acoustic Model Adaptation by Unit Merging on Different Levels and Cross-Level Integration

This paper presents a bilingual acoustic model adaptation approach for transcribing Mandarin-English code-mixed lectures with highly unbalanced language distribution. This includes a adaptation structure, merging of Mandarin and English acoustic units on model, state and Gaussian levels, and a cross-level integration scheme. The corpora tested include two real courses, in which special terminologies were produced in the guest language of English (about 15-19%) and embedded in the utterances produced in the host language (about 81-85%). The code-mixing nature of the target corpora and the very small percentage of the English data made the task difficult. Preliminary experiments showed that unit merging was helpful, merging on lower levels offered more improvements, and cross-level integration was even better. The code-mixing situation considered is actually very natural in the spoken language of the daily lives of many people in the globalized world today.